Remote control system for connected devices

ABSTRACT

Systems and methods utilize a display to provide universal remote control functionality. A menu of options appears on a display of a display device. A user can navigate the various options to control devices that are separate from the display device. The menu may be dynamically configured according to the context in which the display device is being used. The menu may be activated using a remote control device or other device and the menu may include options for controlling one or more devices that are absent from the remote control device.

CROSS REFERENCE TO RELATED APPLICATION

This application claims the benefit of provisional application No.61/480,849, entitled “Remote Control for Connected Devices”, filed onApr. 29, 2011, the full disclosure of which is incorporated herein byreference. This application also incorporates by reference for allpurposes the full disclosures of application Ser. No. 13/077,750,entitled “Apparatus and Method for Configuration and Operation of aRemote-Control System” filed on Mar. 31, 2011.

BACKGROUND OF THE INVENTION

The present invention relates generally to remote control devices andmore specifically it relates to a remote control solution for devices,applications and content by combining a controller with an on-TV userinterface that is adaptable to communicate with and control variousdevices and/or applications.

Remote control devices have been in use for many years. Remote controldevices are utilized to operate various external electronic devicesincluding but not limited to televisions, stereos, receivers, VCRs, DVDplayers, CD players, amplifiers, equalizers, tape players, cable units,lighting, window shades and other electronic devices. A conventionalremote control is typically comprised of a housing structure, a keypadwithin the housing structure for entering commands by the user,electronic circuitry within the housing structure connected to thekeypad, and a transmitter electrically connected to the electroniccircuitry for transmitting a control signal to an electronic device tobe operated.

The user depresses one or more buttons upon the keypad when a desiredoperation of a specific electronic device is desired. For example, ifthe user desires to turn the power off to a VCR, the user will depressthe power button upon the remote control which transmits a “power off”control signal that is detected by the VCR resulting in the VCR turningoff.

Because of the multiple electronic devices currently available withinmany homes and businesses today, a relatively new type of remote controlis utilized to allow for the control of a plurality of electronicdevices commonly referred to as a “universal remote control.” Mostuniversal remote controls have “selector buttons” that are associatedwith the specific electronic device to be controlled by the remotecontrol (e.g. television, VCR; DVD player, etc.). Universal remotecontrol devices allow for the control of a plurality of externalelectronic devices with a single remote control, thereby eliminating theneed to have a plurality of remote controls physically present within aroom.

Conventional universal remote controls are typically programmed usingtwo methods: (1) entering an “identifier code” directly into the remotecontrol, or (2) sampling the control signal transmitted by anotherremote control device. Neither method of programming a universal remotecontrol is efficient and, therefore, many consumers either choose not topurchase a universal remote control or abandon the usage of an alreadypurchased remote control.

Entering identifier codes into a remote control can be time-consumingand difficult for many users. If the user loses the “code book” thatcomes with the universal remote control he or she is oftentimes leftwith a useless universal remote control that he or she is unable toreprogram. Oftentimes a consumer is given four to eight differentpossible identifier codes for a particular brand of electronic device,thereby requiring the user to, through trial and error, determine thecorrect identifier code. Sometimes an individual believes he or she hasentered the proper identifier code since one or two of the commands onthe keypad work only to find out later that one or more commands do notwork, with the electronic device since the proper identifier code wasnot entered.

Also, sampling of control signals is very time-consuming and difficultto ensure proper sampling. An individual must expend significant amountsof time sampling infrared signals from another remote control and savingthese signals within the universal remote control, thereafter assigningthe particular signal to a button on the keypad. This is verylabor-intensive and the results are only as stable as the infrared codesampled.

There are also other problems with conventional universal remotecontrols that make them less desirable for consumers. For example, manyuniversal remote controls have a large number of buttons, many of whichmay never be used, since the manufacturers attempt to have physicalbuttons for each possible command of each possible electronic device.Additionally, even when large numbers of buttons are included in theremote, the programming and compatibility of the remote with new devicesare often limited.

Another problem with conventional universal remote controls is that theelectronic components within these devices are relatively complex andexpensive to manufacture, resulting in an increased cost to theconsumer.

While these devices may be suitable for the particular purpose to whichthey are addressed, from the perspectives of cost, ease of use, andexpandability, they are not optimal. Accordingly, there exist ongoingneeds to provide remote control systems that can be applied to multipledevices in a more intuitive and expandable manner.

BRIEF SUMMARY OF THE INVENTION

The following presents a simplified summary of some embodiments of theinvention in order to provide a basic understanding of the invention.This summary is not an extensive overview of the invention. It is notintended to identify key/critical elements of the invention or todelineate the scope of the invention. Its sole purpose is to presentsome embodiments of the invention in a simplified form as a prelude tothe more detailed description that is presented later.

In view of the foregoing disadvantages inherent in the known types ofuniversal remote controls now present in the prior art, the presentinvention provides a remote control system that uses microprocessor andcommunication functions onboard a display device, such as a television(TV), to allow a user to control multiple devices via a user interface(UI) presented by the display device. This may be accomplished, forexample, using a relatively low-cost remote control that operates theTV, or the like, and provides minimal UI navigation functionality.

Objects of the invention may include the delivery of a full universalremote control solution for devices, applications and content at minimumcost by combining an on-TV UI with a dedicated controller or a standardTV remote with at least one button that is operable to access the on-TVUI. In some embodiments, the on-TV UI may be accessed in other ways,such as through an application executing on a mobile device or tabletcomputing device.

Aspects of the invention may reduce controller complexity by off-loadingfunctionality to an on-TV overlay as well as reduce usage complexity byapplying contextual awareness to provide the most relevant options tothe user.

According to first aspects of the invention, a remote control system isprovided including a display device with a video display, a processorconfigured to interact with a remote control, and a communicationapparatus configured for communicating with one or more other devices.In embodiments, the display device may be, for example, a TV including atuner, audio-visual (AV), high-definition multimedia interface (HDMI),and/or separate video (S-Video) inputs, and, generally, any interfacefor communicating with another device. The display device may alsoinclude wireless communication capabilities, such as WI-FI, BlueTooth,and other capabilities. The display device may also support variousweb-streaming and web browsing modes and, generally, may receiveinformation over a physical wire, fiber optic cable, wirelessly, and/orotherwise.

In embodiments, the processor of the display device may be configured,by programming instructions and the like, to present a user interface(UI) including selectable commands on the display based on capabilitiesof one or more other devices. The display device may be configured toreceive commands for the other device(s) from a remote control; and tocommunicate the received commands to the one or more other devices. Inan embodiment, memory of the display device includes instructionsexecutable by the processor that cause the display device to modify itsdisplay according to various embodiments of the present invention.

In embodiments, the processor of the display device may be furtherconfigured to change the selectable commands based on a current mode ofoperation of the display device, e.g. tuner, AV, HDMI, S-Video,web-streaming, and web browsing modes.

In embodiments, the display device may include selectable modes ofoperation including at least three of tuner, AV, HDMI, S-Video,web-streaming, and web browsing modes. Embodiments may include changingthe selectable commands based on a current mode of operation of thedisplay device and/or displaying different selectable commands in the UIbased on the at least three modes of operation. In embodiments, forexample, different selectable commands may be displayed in the userinterface based on the at least three modes of operation.

In embodiments, the processor of the display device may be configured torecognize a first command received from the remote control, and actionstaken by the processor in response to the first command may be changedfrom a first set of actions to a second set of actions including atleast one different action from the first set of actions. Inembodiments, the first command may be, for example, a display ONcommand, the first set of actions may turn the display ON, and thesecond set of actions may turn the display ON and the other device ON.

In embodiments, the processor of the display device may be configured toautomatically determine whether the other device is ON or OFF, and theUI for the other device may only be presented if the other device is ON.

In embodiments, the processor of the display device may be configured toautomatically determine whether the other device is ON or OFF, and theUI for the other device indicates whether the other device is ON or OFF.In embodiments, the UI for the other device may include a selectablecommand to switch the other device between ON and OFF and vice-versa.

In embodiments, the processor of the display device may be configured toautomatically determine whether the other device is in use and to makean inference of relevant activities for the other device based on acurrent mode of operation of the display device. In embodiments, theselectable commands presented by the UI may depend on the determinationof whether the other device is currently in use.

In embodiments, the processor of the display device may be configured toat least one of activate and update the UI for the other device based onautomated network communication with a remote service provider.

In embodiments, the UI may be configured to allow a user, while watchingthe display device operating in a first mode of operation, to launch anapplication operable in a second mode of operation.

In embodiments, the UI may be configured to allow a user, while viewingcontent on the display device, to accept content-related commands forthe other device, and to communicate the content-related commands to theother device. The content-related commands communicated to the otherdevice may include at least one data element with information based onthe content.

In embodiments, the processor of the display device may be configured todetect a current operational state of the other device and to change theUI for the other device based on the operational state of the otherdevice.

In embodiments, the UI may include a user-selectable and user-definablesearch feature that is operable to search at least content stored on theother device, content available from a remote on-demand service, and/ora scheduled programming service. The processor of the display device maybe configured to perform such searches together, e.g. serially or inparallel, based on a single input from the user.

According to further aspects of the invention, a method of controlling aperipheral device via a display device may include presenting a userinterface (UI) on the display device based on capabilities of theperipheral device. In embodiments, the display device may be, forexample, a TV including a tuner, AV, HDMI, and/or S-Video inputs etc.The display device may also support various web-streaming and webbrowsing modes. Embodiments may include automatically configuring thedisplay device to interact with and/or control the peripheral devicebased, at least partly, on network interaction with a remote serviceprovider.

In embodiments, exemplary methods may include receiving commands at thedisplay device for the peripheral device from a remote control, andcommunicating the received commands from the display device to theperipheral device. The remote control may be any suitable device, suchas a traditional remote control device, a mobile or tablet deviceexecuting a remote control application, and/or generally any device,including handheld devices, from which a user can transmit commands.

According to further aspects of the invention, a computer-readablestorage medium may be provided including executable code for configuringa computer processor to perform any of the methods described herein.

Additional features, advantages, and embodiments of the invention may beset forth or apparent from consideration of the following detaileddescription, drawings, and claims. Moreover, it is to be understood thatboth the foregoing summary of the invention and the following detaileddescription are exemplary and intended to provide further explanationwithout limiting the scope of the invention claimed. The detaileddescription and the specific examples, however, indicate only preferredembodiments of the invention. Various changes and modifications withinthe spirit and scope of the invention will become apparent to thoseskilled in the art from this detailed description.

For a fuller understanding of the nature and advantages of the presentinvention, reference should be made to the ensuing detailed descriptionand accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are included to provide a furtherunderstanding of the invention, are incorporated in and constitute apart of this specification, illustrate embodiments of the invention andtogether with the detailed description serve to explain the principlesof the invention. No attempt is made to show structural details of theinvention in more detail than may be necessary for a fundamentalunderstanding of the invention and various ways in which it may bepracticed. In the drawings:

FIG. 1 is a schematic depiction of an exemplary display system andrelated devices in accordance with at least one embodiment.

FIG. 2 is a depiction of an exemplary screenshot including a deviceselect UI in accordance with at least one embodiment.

FIG. 3 is a depiction of an exemplary screenshot including a commandselect UI in accordance with at least one embodiment.

FIG. 4 is a depiction of an exemplary screenshot including an activityselect UI in accordance with at least one embodiment.

FIG. 5 is a depiction of an exemplary screenshot including anapplication select UI in accordance with at least one embodiment.

FIGS. 6A-6B show example remote control devices in accordance with atleast one embodiment.

FIG. 7 shows a process for providing a menu in accordance with at leastone embodiment.

FIG. 8 shows a process for providing user navigation of a menu inaccordance with at least one embodiment.

FIG. 9 shows a process for providing a menu in accordance with at leastone embodiment.

FIG. 10 shows a process for determining a highlighted menu option when amovie is currently being watched, in accordance with at least oneembodiment.

FIG. 11 shows a process for determining a highlighted menu option whentelevision content is currently being watched, in accordance with atleast one embodiment.

FIG. 12 shows a process of providing a menu of activities shows aprocess for determining a highlighted menu option when a movie iscurrently being watched, in accordance with at least one embodiment.

FIG. 13 shows a process for providing a menu when a user has selected anactivity, shows a process for determining a highlighted menu option whena movie is currently being watched in accordance with at least oneembodiment.

FIG. 14 shows a process for operating at least a subset of a set ofdevices in response to user selection of an activity, shows a processfor determining a highlighted menu option when a movie is currentlybeing watched in accordance with at least one embodiment.

DETAILED DESCRIPTION OF THE INVENTION

In the following description, various embodiments of the presentinvention will be described. For purposes of explanation, specificconfigurations and details are set forth in order to provide a thoroughunderstanding of the embodiments. However, it will also be apparent toone skilled in the art that the present invention may be practicedwithout the specific details. Furthermore, well-known features may beomitted or simplified in order not to obscure the embodiment beingdescribed.

It is understood that the invention is not limited to the particularmethodology, protocols, etc., described herein, as these may vary as theskilled artisan will recognize. It is also to be understood that theterminology used herein is used for the purpose of describing particularembodiments only, and is not intended to limit the scope of theinvention. It also is to be noted that as used herein and in theappended claims, the singular forms “a,” “an,” and “the” include theplural reference unless the context clearly dictates otherwise. Thus,for example, a reference to “a flap” is a reference to one or more flapsand equivalents thereof known to those skilled in the art.

Unless defined otherwise, all technical terms used herein have the samemeanings as commonly understood by one of ordinary skill in the art towhich the invention pertains. The embodiments of the invention and thevarious features and advantageous details thereof are explained morefully with reference to the non-limiting embodiments and examples thatare described and/or illustrated in the accompanying drawings anddetailed in the following description. It should be noted that thefeatures illustrated in the drawings are not necessarily drawn to scale,and features of one embodiment may be employed with other embodiments asthe skilled artisan would recognize, even if not explicitly statedherein. Descriptions of well-known components and processing techniquesmay be omitted so as to not unnecessarily obscure the embodiments of theinvention. The examples used herein are intended merely to facilitate anunderstanding of ways in which the invention may be practiced and tofurther enable those of skill in the art to practice the embodiments ofthe invention. Accordingly, the examples and embodiments herein shouldnot be construed as limiting the scope of the invention, which isdefined solely by the appended claims and applicable law. Moreover, itis noted that like reference numerals reference similar parts throughoutthe several views of the drawings.

The present invention may be used to control and operate variouselectronic devices including but not limited to televisions, electronicgames, stereos, receivers, VCRs, DVD players, CD players, amplifiers,equalizers, tape players, cable units, satellite dish receivers,lighting, window shades and other electronic devices, as well ascomputer applications running on single purpose or general purposecomputers and other electronic devices such as those previouslymentioned. Almost any number of external devices may be controlled bythe present invention as can be accomplished with conventional remotecontrol devices.

FIG. 1 is a schematic diagram of an exemplary display system 100 andrelated devices for practicing various aspects of the present invention.The display system 100 may include, for example, a display screen,various external controls, a microprocessor, random access memory (RAM),read only memory (ROM), one or more speakers, a data bus, and one ormore communications ports and/or devices for receiving and/ortransmitting signals such as, for example, IR receivers/transceivers, RFreceivers/transceivers, network communication modems, HDMI interfaces,Ethernet ports, USB ports, etc. Various ways of incorporating suchcomponents in a display device are known by those of skill in the artand are not exhaustively described herein.

The display system 100 may include a microprocessor, such as ageneral-purpose digital processor, that controls aspects of theoperation of the display system 100. The microprocessor can be asingle-chip processor or implemented with multiple components. Usinginstructions retrieved from memory, the microprocessor may be configuredto control the reception and manipulations of input data and the displayof data, such as UIs, images and video data, and/or the output ofinstructions and/or data to external devices.

The display system 100 may be, for example, a TV configured to receivecommands, e.g. via IR signals and the like, from remote control 110. Theremote control 110 may take various forms, for example, from arelatively simple remote control with ON/OFF, volume and channelfunctions, to universal remote control with an onboard programmableprocessor, memory, display, etc. In some embodiments, the remote controlis a mobile device, tablet computing device, other handheld device, orother device executing an application that enables use of the mobiledevice or tablet computing device to control one or more other devicesin accordance with the various embodiments. According to aspects of theinvention in accordance with at least one embodiment, a relativelysimple and inexpensive remote control 110 may be used by offloading thevariety of functions related to external devices from the remote controlto the display system 100. For example, the remote control 110 may beone designed for use only with the display system 100, i.e. not auniversal remote, and may have limited, or no, onboard programmablefunctionality. In embodiments, the remote 110 may be permanentlyconfigured to transmit only a limited number of command signals. Ofcourse, it should be appreciated that aspects of the invention may alsobe useable with more advanced, programmable, universal remote controls.

The display system 100 may include a network interface to send andreceive data over a network connected to other electronic systems suchas, for example, computer system 126. The network interface may alsoinclude a USB or other connector to connect to myriad peripheraldevices, such as mice, modems, keyboards, storage devices, etc. Aninterface card or similar device and appropriate software implemented byan onboard microprocessor can be utilized to connect the display system100 to an existing network and transfer data according to standard orother protocols including data over a global computer network such asthe Internet. The display system 100 may connect to the Internet 140,which may allow communication with various other services such as, forexample, video streaming service 150, web browsing, device softwareupdate/activation services 152, etc., via one or more separatecomponents (e.g., a computer system, etc.), or directly.

The display system 100 may communicate directly or indirectly withvarious other electronic devices, such as, for example, a DVD player120, Audio/Video Receiver (AVR) 122, gaming system 124, and computersystem 126. Such communications may be performed by any well-knowncommunication system that allows communications with external electronicdevices, e.g. infrared (IR), consumer electronics control (CEC),Internet protocol (IP), unidirectional and/or bi-directional radiofrequency (RF), visible light, light outside of the visible spectrum,ultrasonic, and various other means. In embodiments, such communicationsmay be performed according to predetermined control protocols for theexternal devices, e.g. according to remote control and/or controllerprotocols of the external devices. The display system 100 may beconfigured to obtain such protocols, for example, from remote serviceprovider 152 over the Internet 140, local programming input, a localcomputer network, direct communication with external devices 120, 122,indirect communication with device 124, and/or receiving a “signalsample” from another remote control.

The display device 100 may include memory and one or more processorsthat collectively execute instructions stored in the memory. The one ormore processors may operate a background application that collectsinformation and performs operations in accordance with the presentdisclosure. For example, the one or more processors may enable thedisplay device to implement a background application that determinesvarious menus and presents the various menus while the display devicesimultaneously displays other content. The memory may additionally storevarious information that allows for various determinations to be madefor the presentation of dynamically determined menus in accordance withthe various embodiments. For example, the memory may store one or moretables or other data structures that include various information aboutvarious devices that are used with the display device 100. In addition,the display device (or other device) may be configured to interact withexternal devices over several mediums, including, but not limited to,IR, integrated IR, external blaster accessory, IP, CEC over HDMI, IPover home network, RF4Ce, Bluetooth, and the like.

The present invention is well suited for use with a global computercommunication network such as the Internet 140. However it can beappreciated that, as future technologies are created, various aspects ofthe invention may be practiced with these improved technologies. Inaddition, various wireless technologies provide a suitablecommunications medium for operating aspects of the present invention.

Remote control 110 may include various functionality, typicallyincluding some form of wireless communications means and various inputbuttons that allow the user to execute one or more commands to thedisplay system 100. In embodiments, remote control 110 may be, forexample, a smart phone, PDA, tablet computing device, or other hand-helddevice with the ability to transmit at least some wireless commandsignal.

According to aspects of the invention, the display device may beconfigured to provide various menus and UI for the control of externaldevices such as those shown in FIG. 1. For example, a first screen 200shown in FIG. 2 may be displayed on the screen of the display device100. Screen 200 may include a background 210, which may be, for example,a TV broadcast, a streaming video, a video game, or web browser view. Inone embodiment, the UI is an overlay upon such existing content on thedisplay. Upon activation of the UI, screen 200 may also includenavigation bars 220, 230, and selection bar 240, or other UI barsdescribed herein, which may be responsive to commands received fromremote 110. For example, remote control 110 may include “navigation”arrows that allow the user to move between bars 220, 230 and 240, andwithin selection bar 240. The remote 110 may also have a “select” buttonto activate a highlighted bar or selection. Generally, a UI may bedesigned such that it can be navigated with other input mechanisms,including mice, track pads, touch screens on mobile or other devices,and the like.

As shown in FIG. 2, the device selection bar 240 may be considered a“Device” UI and may include UI functionality for selecting variousdevices, including devices external to the display device 100. Forexample, the user may select operation of the “TV” icon for the displaydevice itself, the “DVD-1” icon for the DVD player 120, the “AVR” iconfor the AVR 122, or the “Game” icon for video game 124. One, more, orall selectable options on the UI (or other UIs) may be options that areincapable of being selected using a dedicated input of a remote controldevice. For example, a remote control used to navigate the UI shown inFIG. 2 may not have any inputs capable of being used to select any ofthe options in the menu other than one or more buttons that are used toselect a currently selected option in the UI. For instance, a remotecontrol may not have a button that allows selection of “DVD-1,” asillustrated in FIG. 2. It should be noted that the devices shown in thedevice UI are provided for the purpose of illustration, and differentdevices may be shown in a device UI. Further, in place of one or moredevices, one or more icons may correspond to an input of the display orother device, for example, if the exact nature of the device is notknown. The devices or other icons shown may correspond to those devicesthat are actually in communication with the display device, althoughother configurations are considered as being within the scope of thepresent disclosure. It should further be noted that, although variousscreens described herein may be referred to for convenience and ease ofunderstanding as “Device UI”, “Control UI”, “Activities UI” etc., itshould not be inferred that each necessarily represent a discreetlyoperating user interface engine. Rather a single code engine, or variouscombinations of any number of code engines may be used to implement suchfunctionality.

Display system 100 may be configured to show, for example, only devicesdetected by the display system 100, devices selected and/or input by auser, devices that have been previously detected and/or input that arecurrently ON, devices that are currently in use, devices that have beenused recently, or other predetermined or user-selected criteria. Incases where all detected and/or input devices are not currentlydisplayed, the device selection bar 240 of FIG. 2, or other UI, mayinclude additional icons, and the like, for forcing display of theun-displayed devices, e.g. to allow turning a device ON, controlling aseldom used device, etc. In embodiments, the UI may show commands forall devices attached to the display device, whether they are in use ornot. This way, the user can turn ON/OFF an external device from the UIdirectly, thus minimizing the number of buttons on the controller oralleviating the need for a LCD on the controller.

The UI including bars 220, 230, and 240, as well as other UI and screensdescribed herein, may be displayed, for example, manually, e.g. bypressing a single button on the remote 110, or automatically, e.g. whena new external device is detected, when an external device is turned ON,when predetermined content is detected by the display device 100, and/oraccording to an application running on display device 100 or an externaldevice. In embodiments, the remote control 110 may include a dedicatedcontrol (e.g. a hard or soft button) that causes a UI to appear on thescreen of the display device. Likewise, when the user is done navigatingthe UI, it may be closed by pressing the dedicated button or after atimeout.

In embodiments, the display device 100 may be configured to infer whatexternal devices are in use and/or relevant activities by tracking, forexample, which display device input is selected. Such processes mayinclude an initial identification and association of the particularexternal devices and input sources, e.g. as an initial setup processflow, through CEC device identification, or in other ways.

When the user selects, via remote control 110, either of navigation bars220, 230, the displayed UI may be changed to present one or moredifferent selection bars and navigation bars. For example, as shown inFIG. 2, selecting navigation bar 220 may present an “Activities” bar, orselecting navigation bar 230 may present a “Web Apps” bar, discussedfurther herein.

When the user selects, via remote control 110, a device icon from deviceselection bar 240, a variety of functions may be implemented. Forexample, selecting the device icon “TV” may make available commandfunctions for the display device 100 itself, selecting the device icon“DVD-1” may make available command functions for the DVD player 120,etc. Selecting an icon for an external device may also cause, forexample, an input and/or operating mode of the display device 100 tochange to an appropriate mode for the selected device, if applicable,and/or cause predetermined commands to be sent to the selected device,e.g. a power ON command, etc. As shown in FIG. 2, the DVD-1 icon ishighlighted. In this particular example, the DVD-1 icon is highlightedby a rectangle that surrounds the icon, although the icon, or any icon,can be highlighted in any manner. Generally, highlighting may beperformed in any manner that emphasizes an item being highlighted. Theuser may have used navigational controls of a remote control device tocause the DVD-1 icon to be highlighted. The DVD-1 icon may have beenautomatically for highlighting for various reasons discussed below. Forexample, the display device (or another device) may have determinedthat, when the user caused the device UI to appear, the DVD-1 icon wasmost likely to be selected by the user and/or it was determined that theuser was utilizing a DVD player at a DVD-1 input when the user took anaction that caused the UI to appear. It should be noted that variousoptions of a UI may be de-emphasized. For example, options for devicesthat are not currently operable, due to a power-off state, a lack ofconnection, or another reason, may be displayed with less brightness,moved to a less prominent place on the UI, or otherwise displayed in amanner that is emphasized less than other devices which are currentlyoperable. As an illustrative example, if a user often uses a gamingsystem but a connection to the gaming system is not detected, an iconfor the gaming system may be displayed less prominently than if the aconnection to the gaming system is connected. In other embodiments, asmentioned above, various options of a UI may not be presented. Forexample, options for devices that are not currently operable, due to apower-off state, a lack of connection, or another reason, may not bepresented.

By way of further example, selecting a device icon from device selectionbar 240 may cause the UI to change and present an appropriate “Command”UI as shown in FIG. 3. As shown in FIG. 3, the screen 300 may includethe same background 210 as shown in FIG. 2, or a new backgrounddepending, for example, on whether a mode of the display device 100 waschanged based on the input command. For example, if the background isfrom a video, the background 210 may be updated according to a currentframe of the video. The navigation bar 330 may be a new navigation barbased on the relative position of the “Command” UI with respect to otherUI. In this case, navigation bar 330 presents a selectable option tonavigate to an “Activities” UI discussed further below. In embodiments,the user may define, and/or the display device 100 may automaticallydetermine, a hierarchy for the various UI screens such that the user maynavigate most easily between their preferred or often used UI screens.Likewise, the UI that is initially displayed to the user may bepreconfigured by the user and/or based on other detected factors such ascontent being viewed, detection of a new device, etc.

As also shown in FIG. 3, a control selection bar 340 may include variouscontrol icons specific to a selected device, e.g. DVD player 120. Suchicons may be based on predetermined criteria related to the type ofdevice, and/or may include operations specifically determined for thedevice model, manufacturer, etc. For example, the display system mayhave pre-loaded commands for commonly used devices, and/or specificmakes and models of devices, and/or may be configured to receive oraccess device command information as needed for detected or inputdevices. Such updates may be performed, for example, upon detecting anexternal device or based on a user adding an external device to becontrolled. The display or other device may dynamically determine whatcommands are to be displayed, as described in more detail below.

In the example shown in FIG. 3, command selection bar 340 includes,among others, icons for “Replay”, “Rewind”, “Play”, “Forward” and “Skip”for instructing DVD functions. Command selection bar 340 may alsoinclude a “Power” icon for turning the selected device ON and OFF. Uponselection of such icons, and others appropriate to various devices undercontrol, the display device 100 may communicate appropriate commands tothe selected device via, for example, a wireless command signal that theselected device is configured to receive and respond to, or via HDMIconnection and the like. As shown in FIG. 3, a “play” command ishighlighted. As above, the user may have navigated to the play commandor the display device (or another device) may have determined that theplay command was the most likely to be selected by the user. As oneillustrative example, a video may be in a paused state and, as a result,the display or other device may have determined that the next commandselected is likely to be play.

The UI, and particularly command UI such as shown in FIG. 3, may also beaware of the current state of the external devices in use, and maychange an icon representing a function state, in order to, for example,minimize the need for navigation of the UI. For example, if a live TVbroadcast is paused by the user (such as by a digital video recorder(DVR)) via a command selection bar similar to that shown in FIG. 3, theUI may automatically highlight for activation the “Play” command iconand thus avoid the user having to manually navigate the menu of DVRcommands. As another example, if a user has opted to mute an externaldevice such as an AVR, an icon presented on the TV, such as a volumeicon, may be change to indicate that the AVR device is indeed muted. Inembodiments, such changes may be based, for example, on two-waycommunication with the external device, such as by HDMI, that allows thedisplay device to confirm that certain commands have been executed.

Command selection bar 340 may also include icons for commands that arenot specifically directed to the selected device, but that may beotherwise related to the selected device. For example, upon selecting aDVR or other playback/recording device while viewing a video program,movie, etc., the device UI may bring up commands for the selected deviceand other related commands for the display device itself, such as inputsource, mode, etc. Such functions may be particularly useful when it isnot immediately clear what a particular device is being selected forbased on currently viewed content or other factors. For example, whilewatching video programming, a user selecting a DVR device might want torecord the currently viewed content, switch to watching a pre-recordedshow from the DVR, view a menu of available content from the DVR, etc.Providing the user with an intelligent selection of command icons mayhelp improve the ease of use as well as inform the user that certainoptions are, or are not, available.

Command selection bar 340, and the like, may also be configured topresent sub-groups of commands, for example, in an order of most likelyusage, e.g. “Volume” may be accessible before “Inputs”, which may beaccessible before “Sound modes” on an AVR, etc. The UI may presentcommands within a sub-group in an order of most likely usage, e.g.“Stereo” accessible before “Surround 5.1”, which may be accessiblebefore “Jazz” or “Mono” in a “Sound modes” sub-group. The order of thecommand icons may be defined, for example, based on tracking usage of acommunity of users, on tracking usage of individual user, and/or basedon the nature of content being viewed. For example, if the user iswatching a concert video, the display device may be aware of, orotherwise determine through programming guides and the like, the natureof the content, and prioritize displayed command icons based on expectedinput for watching concert videos or related content.

The user may also select the navigation bar 220 to switch to an“Activities” UI shown in FIG. 4. As shown in FIG. 4, the screen 400 mayinclude the same background display 210 without interruption, andpresent new navigation bars 420, 430 and activity selection bar 440.Activity selection bar may be configured to present variouspredetermined and/or dynamic activity icons representing categories ofactivities. For example, as shown in FIG. 4, activity icons may include“Watch TV”, “Watch Movie,” “Listen to Music”, and “Play Game”. Theseicons may be dynamically displayed according to, for example, specificdevices and/or applications that have been detected or input for controlby the display device 100. In the example shown in FIG. 4, “Watch TV”may be a permanent activity icon related to an inherent capability ofthe display device 100, whereas “Watch Movie”, “Listen to Music”, and“Play Game” may be a dynamically loaded activity icons based on thedetection and/or input of DVD player 120, AVR 122, and video game 124,respectively, or other online available service.

In embodiments, the selection of an activity associated with particulardevices may be configured to send commands to one or more devices undercontrol, e.g. to turn on and/or adjust settings of the device(s)associated with the activity, etc. The selection of the activity mayalso initiate internal commands to the display device 100, such asswitching an input source, a viewing mode, sound settings, etc. By wayof further example, selecting a “Watch Movie” icon may turn on a moviesource device or application, such as a DVD player, adjust sound settingof the AVR and the like, dim home lighting, and adjust the brightness ofthe display device. As shown in FIG. 4, a “Watch TV” icon ishighlighted. As above, the user may have navigated to the icon or anautomated process may have determined that the icon should behighlighted.

The display device 100 may also be configured such that the selection ofan activity icon from the activity selection bar 220 switches the UI toa device UI, similar to that shown in FIG. 2, or other UI, displaying atailored group of controllable device and/or application icons relevantto the selected activity. For example, selecting a “Watch Movie” iconmay bring up a UI with all available devices and/or applications thatthe display device can control with the ability to feed movies to thedisplay device, as well as any other devices that may be relevant tomovie watching, e.g. home lighting etc. Other configurations are alsopossible, such as drop down menus and the like, for selecting a desireddevice under a particular activity icon.

In embodiments, the UI may also include a screen for applicationselections, such as web-based and/or computer-operated applications. Anexample of such an “Application UI” is shown in FIG. 5. As shown in FIG.5, a screen 500 may be presented with an ongoing background presentation210, and include a navigation bar 520 to a Device UI, and an applicationselection bar 540. Application selection bar 540 may include variousapplication initiation icons, such as, for example, web browsers,specific web sites, web streaming services, etc. Accordingly, the UIpresented on screen may enable a user to, for example, interact withsocial networks, or other web-based applications, and the like, whilecontinuing to view content on the display. For example, a selectableicon may be used to open chat windows and the like in a portion of thescreen, along with an on-screen keyboard that the user can navigate andinput text. In embodiments, the applications opened may be automaticallyinformed of content-related information, such as, for example, detailsof a show being watched on the screen which may in turn be used to guidethe user to a desirable social network and/or online chat discussion.

In embodiments, the display device 100 may be further configured toprovide support to online lifestyle functions. For example, based on aselected “activity,” the UI may display a limited number of specificcommand icons for access to applications which the user can interactwith in the context of watching a specific piece of content. This mayinclude, for example, by selecting activation of an online postingservice while watching a particular show, the display device canautomatically access the online posting service via an IP device andpost a message such as “User XYZ is watching show ABC,” where “user XYZ”and “show ABC” are variables that can be replaced with a user identifier(such as username) and the title of a show, respectively. Additionallyand/or alternatively, the UI may present the user with a list ofpreconfigured posts, which may be based on content being watched, toautomatically transmit when selected via the UI. Therefore, at the pushof a single button on a TV UI, a user could automatically post a messagesuch as “I like show ABC” on a social networking site (e.g., Facebook®,Twitter®, Etc., to name just a few), while still viewing content on theTV.

As noted above, the particular application selection icons presented maybe tailored based on a type of content being viewed. For example, the UImay be configured to provide ready access to sources of information,e.g. from the Internet, related to the content currently consumed. Onesuch option may be that, when watching a movie, the UI presents iconsautomatically, or manually, determined to be relevant to movie content,e.g. movie database websites, movie review websites, etc. Inembodiments, the display system may be configured to automaticallyextract specific content from websites and the like based on the typeof, or specific, content being viewed. The UI may be configured to, forexample, open an overlay UI, or a picture in picture, showing a directorfilmography, actors' biographies, movie synopsis, user reviews, userratings, etc. for a movie being watched. Such functions may be enhancedby including content-related information in commands sent to thewebsites, e.g. automatically including the name of the movie beingwatched to a movie database website upon selection of the movie databasewebsite icon.

By way of further example, when watching a sports game, the UI may beconfigured to provide quick access via a limited number of command iconsto a website or other service with tabulated information about the gamebeing watched, results of other games, and/or player/team statistics. Asanother illustrative example, a normal view of a football game may beshown prominently on the screen of the display device and a birds-eyeview of the field may be shown in a smaller portion of the screen.

Such applications and services may be implemented and/or accessibledirectly by the display device 100 or may be initiated and/or executedvia an external device. For example, for TVs with built-in browsingand/or streaming functionality, the TV may initiate and receive data,video and/or audio streaming via the Internet and the like. In othercases, an intermediate device, such as a video game console 124, may beused to access streaming content, e.g. that is delivered from serviceprovider 150 and the like, to the display device 100 via the video gameconsole 124. As also shown in FIG. 1, configurations where a video game,or other device communicates to the display device 100 via anintermediary device, such as AVR 122, are also possible. Other examplesof intermediate devices include set-top boxes, content appliances suchas the Logitech Revue available from Logitech®, and other devices mayparticipate in various embodiments of the present disclosure.

It should also be noted that the UI presented on the display device,such as a TV, may enable a user to launch applications integrated on thedisplay device, e.g. a video streaming service, without the need foruser to stop watching TV and separately launch the application portal inorder to switch modes of operation, e.g. from watching broadcast TV tovideo streaming or playing an onboard TV game such as may be downloadedfrom TV application stores and the like.

The display system 100 (also referred to herein as the display device100) may also be configured to allow a user, while viewing content onthe display device, to accept content-related commands for an externaldevice or application, and to communicate the content-related commandsto the external device or application. The content-related commandscommunicated to the external device or application may include at leastone data element with information based on the content. For example, theUI can present the user with the option to queue a show currently beingwatched on broadcast TV to an on-demand video streaming service queue,at the selection of a single icon in the UI. Such commands may include,for example, registering a time stamp where the live show is stopped, inorder to begin it at another time via an on-demand service preciselywhere the user left off.

In embodiments, the UI may also include a user-selectable and/oruser-definable search feature that is operable to search, for example,content stored on an external device, content available from a remoteon-demand service, and a scheduled programming service. The processor ofthe display device may be configured to perform such searches together,e.g. serially or in parallel, based on a single input from the user.Such searches may be presented to the user and/or performed with contentoptions related to content currently being watched, e.g. a “show me morelike this” search, etc. Additionally, search results may be presented inan order related to the currently viewed content or type of “activity”as described herein. For example, if a user is watching live TV, whichmay be interpreted as a “Watch TV” activity, live show search resultsmay be presented before recorded shows, which may in turn be presentedbefore future live shows, which in turn may be presented before Internetshows (which may require devices different from those needed for thecurrent activity).

In various embodiments, UI services for controlling external devices maybe turned on remotely, for example, by cable TV providers, remoteservice providers, etc. that may send activation signals to the displaydevice. Activation may thus allow for additional functionality to beadded to a remote control device, as part of a pay service or the like.For example, before activation, a power button on the controller 110 mayonly turn the display device 100 on. After activation, which may, butdoes not necessarily, include payment of a fee, and/or registration witha service provider, etc., the power button on the controller 110 mayturn the TV ON and trigger an onboard application to turn an AVR and STBON, and switch TV and AVR inputs. Thus, the same power button on theremote 110 may be effectively transformed from a “TV ON” button to a“Watch TV” activity button with multiple related functionality.

In embodiments, web pages, or other accessible support services, may beprovided for configuring exemplary display devices for use with variousexternal devices. For example, web pages, or the like, may be presentedto a user on the display device allowing for entry of deviceidentification for external devices under control of the display device.Information relating to the type, brand and model of the external devicemay be entered into a web page that may access a database to determineconfiguration and/or control data. Such identification, configurationand/or control data may include, for example, product information, type,brand, model, year, communication type, and signal configuration data.

The display device may also be configured with a database and/orconnectivity that allows for the determination of an electronic deviceby one or more signal samples from the corresponding remote control. Thedisplay device may access an onboard, or remote, database that allowsfor the determination of what inputs and outputs are on the electronicdevices associated with given control signals provided by thecorresponding remote control. An onboard database may be updated atperiodic intervals regarding updated information or regarding newelectronic devices on the market.

One method of operating the display device to identify an externaldevice to be controlled is to manually sample a signal emitted from aremote control corresponding to the electronic device to be controlled.Prior to sampling the signal, the user may select a “Sample” icon on theUI which will place the display system in sample mode for receiving oneor more sample signals via one or more communication mechanisms such asan IR receiver, etc. Signal sampling has been performed within theremote control industry for years and is well known to those skilled inthe art particularly with IR sampling. No further discussion of signalsampling is required as the same is readily apparent in the art.

The display device, and/or a remote service provider, may analyze thesample data to determine a type, brand and/or model of each of theelectronic devices that are controlled by the corresponding sampledsignal. Once the type, brand and/or model of each of the electronicdevices is determined, the display device or the remote service providermay generate configuration data that is loaded and/or downloaded to anactive memory of the display device. The configuration data configuresthe display system so that it is able to control all of the externalelectronic devices as, for example, a remote control manufactured forthe external device would. The user may then access various UI asdescribed herein to control the associated external device. Inembodiments, the display system may also prompt the user via the UI foradditional information that may be used to create a personalizedconfiguration.

As mentioned previously, the user may avoid sampling the signal fromeach of the remote controls and instead directly enter productinformation into the UI, or other accessible remote service, such as aweb page. The user may enter, for example, relevant product informationsuch as but not limited to device type, e.g., VCR, television, DVDplayer, home lighting device, etc., brand and model.

Once the display device or remote service has determined the type, brandand model of the input external device, the display device or remoteservice may generate corresponding configuration data that is loaded ordownloaded to an active memory of the display system, and may be used bythe display device to control the input external device as discussedpreviously.

The display system may also be configured to automatically determine atype, brand and/or model of an external device via communication withthe external device itself. For example the external device may have atwo-way communication capability, such as HDMI or USB ports, etc., andmay exchange information with the display device sufficient for thedisplay device to recognize the external device's type, brand and/ormodel, similar to “plug-and-play” type recognition.

As noted above, various embodiments of the present disclosure allowusers to control a UI using various devices. In some embodiments, usersare able to control a UI using a remote control device. FIG. 6Aillustrates an example remote control device 600 that may be used inaccordance with various embodiments. In this particular example, theremote control device 600 is simpler than typical remote controldevices. For example, although a TV and/or other device may havenumerous functions, the remote control device 600 in this example haseight tactile inputs, although a typical TV or other device may havemany more than eight possible functions that can be performed. It shouldbe noted that the remote control device 600 illustrated in FIG. 6A isprovided for the purpose of illustration and that a wide variety ofremote control devices are considered as being within the scope of thepresent disclosure. As just a few examples, a remote control device mayinclude fewer or more tactile inputs than illustrated and/or may userother input mechanisms for user input, such as a touch screen, touchpad, and/or other device. As discussed below, other devices, such ashandheld computing devices executing applications for remote controllingmay be used to control a UI.

In this example, the tactile inputs of the remote control device 600 arein the form of various buttons that correspond to various functions of aTV or other device. For example, the remote control device may have apower button 602 that, when selected, causes a TV and/or other device totoggle between a power-on state and a power-off state. Selection of thepower button 602 (or any other button and/or combination of buttons) maycause the remote control device 600 to transmit an appropriate signal(or collection of signals) to the TV and/or other device. Receipt of asignal from the remote control device may cause the TV and/or otherdevice to perform a corresponding function.

In this example remote of control device 600, other tactile inputscorrespond to frequently used functions of a TV. For example, the remotecontrol device 600 may include a volume button 604 and a channel button606. In this example, depending on where the user selects the volumebutton 604, the volume of the TV (or other device providing audiocontent, such as an AVR) may be adjusted accordingly. Similarly,depending on where the user selects the channel button 606, the TV (orother device, such as a set top box (STB)) may tune to a next orprevious channel in a sequence of channels. It should be noted thatvarious buttons on the remote control device 600 may cause differentfunctions to happen in different contexts. For example, when a user iswatching TV, the volume button 604 and the channel button 606 may causeoperations as described above. However, in other contexts, operationsperformed as a result of selection of the volume button 604 and/orchannel button 606 may differ. As an example, in some contexts,selection of the volume button 604 and/or channel button 606 may causecorresponding navigation to occur in an on-screen menu. The signal sentfrom the remote control may (but does not necessarily) remain the sameregardless of the context. Accordingly, the signals received from theremote control device 600 may be interpreted differently depending onthe context.

Other common tactile inputs that may be on the remote control device 600include a mute button 608 and a “last” button 610. Selection of the mutebutton may cause a TV and/or other device providing audio to togglebetween a state where audio is provided and a state where audio ispartially or completely muted. Selection of the last button 610 maycause the TV or STB (or other device) to tune to an immediatelypreviously viewed channel. Other common inputs that may be included inthe remote control device 600, but that are not shown, include numericalkeypads, cancel buttons, and the like. The remote control device mayalso include a QWERTY keyboard and/or other input devices in someembodiments. The remote control device may also include a display thatmay or may not be a touch screen. The display may include one or moresoft buttons for selecting options of UIs presented on the display.

In an embodiment, the remote control device 600 includes an Internetbutton 612. Selection of the Internet button may cause Internet contentto appear on the TV display. The Internet content may occupy thecomplete display, a portion of the display, and/or may overlay othercontent, such as described above. In addition, the tactile inputs of theremote control in this example include a navigational input 614 which,as illustrated, includes a navigational portion 616 and a selectionbutton 618. The navigational buttons may be used by a user to navigatethe various UIs, such as the UIs displayed above. The selection button618 may be used by a user to select a highlighted option on a UI.

In an embodiment, the remote control device 600 includes a menu button620. Selection of the menu button 620 may cause a UI, such as a UIdescribed above, to appear on the display device. For example, if theuser is watching a movie on the display device and presses the menubutton 620, an appropriate UI may appear overlaid on the display. Themovie may continue while the UI is overlaid on the display. The user maythen navigate the menu, such as by using the navigational input and makea selection of a highlighted item of the UI by pressing the selectionbutton 618. For example, referring to FIG. 3, if the user selects thehighlighted play option when a movie or other content is in a pausedstate, the movie or other content may resume playing. If, however, theuser selects the right arrow of the navigational portion 616 of thenavigational input 614, the fast-forward option may become highlightedinstead of the play option and, if the user then selects the selectionbutton 618, the movie or other content may play at a faster pace. If theuser presses the down arrow of the navigational portion 616 of thenavigational input 614, the activities selection bar may becomehighlighted and, if the user selects the down arrow again or theselection button 618, the activities UI may appear in place of thecommand UI 330. Other possibilities are clear from context and,therefore, are not elaborated upon further.

As noted, the remote control device 600 is provided for the purpose ofillustration and a wide range of variations are considered as beingwithin the scope of the present disclosure. For example, in oneembodiment, a remote control device is simpler than that shown in FIG.6A, having only a menu button and navigations keys, such as left, right,up, and down keys and a selector key. Also noted, the functionality ofthe remote control device 600 may be provided in other ways. Forexample, in an embodiment, a mobile device (such as a smart phone ortablet computer) is used to generate signals that are used to obtain andnavigate menus using any communication method (such as Wi-Fi) that themobile device is configured with. A remote control application mayexecute on the mobile device. Execution of the remote controlapplication may cause a representation of a remote control device toappear on a touch screen of the mobile device. A user may, therefore,operate the mobile device as a remote control by touching the touchscreen at locations that correspond to displayed buttons or otherinterface elements.

In addition to the above, other variations are also considered as beingwithin the scope of the present invention. Many frequently used commandssuch as Volume Up/Down, Mute, Channel Up/Down, Play/Pause, etc. are bestaccessible from the remote control (as opposed to a UI on a display).However, as some functions are applicable to several devices, e.g. bothPVR and DVD support transport commands (Play/Pause), the correct IR orCEC commands may be automatically sent to the content source devicecurrently active, which may be tracked by a device that configures theUIs and/or the remote control device itself. For example, when a userpresses a Volume UP option of the remote control device 600, acorresponding signal may be configured and sent to an AVR if the audiois provided by the AVR or a corresponding signal may be configured andsent to a TV if the TV is providing the audio. In this manner, the needfor the user to program the TV remote or use the TV UI to accessdevice-specific commands that are frequently used may be partially orcompletely avoided. Based on the content source device selected, andknowing whether it is connected to the TV or the AVR, the deviceproviding the UI (or another suitable device can for instance determinewhether to send: a Play/Pause command to either the DVD or STB; a volumecommand to the AVR, send it to the TV or process it within the TV.

In an embodiment, the remote control device 600 is a remote controldevice provided with a display device (such as a TV remote control). Theremote control device may include an INPUT button (not illustrated) thatinvokes the Device UI and as well as the TV inputs menu. The remotecontrol device 600 may also include a FAVORITE button on the TV remoteshould invoke an appropriate UI with a Favorite Content menu or aFavorite Content menu that is specific to the current activity orcurrently selected content source, e.g. Favorite Channel for “Watch TV”or STB, and Favorite Internet Radio stations for “Listen to DigitalMusic” or DMA. In addition, when a TV UI is displayed, the correct IR orCEC command may be sent to the device currently selected in the TV UI,if commands are sent from the remote control device. For instance, aVolume Up/Down command may be sent, by the remote control or otherwise,to either the TV or AVR depending on which device is currently presentedin UI. This allows user to control all functions of every device, evenif the TV UI does not include a command but the command is accessible onthe remote control.

As noted, other devices may be used in accordance with variousembodiments of the present disclosure. FIG. 6B, for example, shows anillustrative example of a mobile device 600B which may be used tocontrol one or more consumer electronic devices in accordance with thepresent disclosure. In this particular example, one or more processorsof the mobile device 600B operate according to executable instructionsfor an application that enables the mobile device 600B to display a UIon a touch screen 601B of the mobile device. It should be noted that,while FIG. 6B shows a device with a touch screen, mobile devices used inaccordance with the present disclosure may not have a touch screen, butmay have another display that is navigable using input devices otherthan a touch screen.

In an embodiment, the mobile device 600B displays a UI that includes aplurality of inputs for controlling a UI on a display device of anotherconsumer electronic device (such as a television). The UI of the mobiledevice, as illustrated, includes buttons such as those described abovein connection with FIG. 6A. For instance, as illustrated, the UI of themobile device in FIG. 6B includes a power button 602B, a volume button604B, a channel button 606B, a mute button 608B, a “last” button 610B,an Internet button 612B, a navigational input 614B that includes anavigational portion 616B, a selection button 618B, and a menu button620B. The buttons of the UI of the mobile device 600B may be used in amanner such as described above in connection with similarly-namedcounterparts in FIG. 6A. In this example, however, the buttons of the UIof the mobile device 600 are not physical buttons as illustrated in FIG.6A, but virtual buttons. A button on the UI of the mobile device, forexample, may be selected by touching the touch screen 601B at a locationwhere the button is displayed on the touch screen. Selection of a UIelement on the UI of the mobile device may cause one or more otherdevices to operate accordingly, such as described above.

It should be noted that the UI shown in FIG. 6B is provided for thepurpose of illustration and that variations are considered as beingwithin the scope of the present disclosure. For example, variationsdiscussed above in connection with FIG. 6A may be virtually presented onthe UI of the mobile device 602. Further, the display of the mobiledevice 602 may not resemble controls of a traditional remote control, asillustrated in the drawing. The UI may have numerous variations and, insome instances, the UI of the mobile device may resemble the UIdisplayed on another device (such as a television). The UI of the mobiledevice may, for example, change state according to changes in state ofthe UI of the other device.

It should further be noted that the various UI menus of FIGS. 2-5 areprovided for the purpose of illustration and a wide variety ofvariations are considered as being within the scope of the presentdisclosure. For example, a UI may present all related commands groupedinto a sub-menus, such as “Inputs”, “Sound modes”, etc. A UI may presentsub-groups of commands in an order of most likely usage, e.g., Volume isaccessible before “inputs”, which is accessible before “Sound modes” onan AVR. A UI may present commands within a sub-group in an order of mostlikely usage, e.g. “Stereo” is accessible before “Surround 5.1”, whichis accessible before “Jazz” or “Mono” in a “Sound modes” sub-group.Order may be defined based on tracking usage of community of users, oron tracking usage of individual user, but also based on nature ofcontent in embodiments where one or more devices are configured to beaware of what content is being played and where the one or more deviceshave access to a source of information describing the content, such as aTV Guide, which may be provided from a remote computer system over, forinstance, the Internet. Examples of how this may be accomplished aredescribed in application Ser. No. 13/077,750, entitled “Apparatus andMethod for Configuration and Operation of a Remote-Control System” filedon Mar. 31, 2011, referenced above.

As other examples, a UI may include a search interface that allows forinteraction with various devices to, for example, enable searching forrecorded content on a DVR, live content available for user's channelline-up, or services built into a TV or other device, such as a moviestreaming service application. Such embodiments may allow a user toinput credentials for accessing various data stores and may includeintegration with a remote computer system operating to provide suchinformation. As an enhancement, search results could be presented in anorder that is related to the current activity. For instance, if user iswatching live TV, live shows may be presented more prominently thanrecorded shows (such as higher in search results), which in turn arepresented more prominently than future live shows, which in turn arepresented more prominently than internet shows (which may requiredevices different from those needed for the current activity).

As noted, the UIs presented in FIGS. 2-5 are illustrative examples ofjust a few possible UIs that may be presented on a display. UI optionsmay be favorites driven. For example, regardless of the video sourcecurrently selected, if the user selects a Favorite channel in the TV UI(i.e. a UI on a display device), an appropriate device may dynamicallycompute and automatically send the correct sequence of IR or CEC codessuch that the STB is ON, DVD is OFF, AVR is ON (if needed), the right TV(and AVR if needed) inputs are selected and the STB is tuned to thedesired channel. UI options may also be shortcut-driven. For example,regardless of the video source currently selected, if the user selectsan Application shortcut, e.g. an Internet content streaming option inthe TV UI, the Harmony engine dynamically computes and automaticallyfires the right sequence of IR or CEC codes such that the STB is OFF,DVD is OFF, AVR is ON (if is for surround sound through HDMI returnaudio channel), the right AVR input is selected (if needed) and theright application is launched through a SW call to the internal TV API.UI options may also be power-driven. For instance, if no video source isactive (only TV is ON), when the user powers ON a content source (e.g.DVD, STB, gaming console, CD) in the TV UI, an appropriate device maydynamically compute and automatically send the right sequence of IR orCEC codes such that the content source is powered ON, the AVR is poweredON (if required) and the right TV (and AVR if needed) inputs areselected. The same may apply if a first content source device is activeand the user powers on a second one from the TV UI. In this case, thedevice computing and sending the codes may also powers OFF the firstdevice. As yet another example, UI options may be input-driven. Forexample, if no video source is active (only TV is ON), when the userselects in the TV UI a TV or AVR input mapped to a specific contentsource device (e.g. DVD, STB, gaming console, CD), a device maydynamically computes and automatically send the right sequence of IR orCEC codes such that the content source is powered ON, the AVR is poweredON (if required) and the right TV (and AVR if needed) inputs areselected. The same may apply if a first content source device is activeand the user selects a different TV or AVR input from the TV UI. In thiscase, the device computing and sending the appropriate codes may powerOFF the first device.

Various embodiments may also be function driven. For instance, users maybe able to trigger an activity by pressing a button on their remotecontrol that is uniquely or most likely associated with an activity. Assome examples, if a user presses the POWER button on a TV remote or anyother universal controller, e.g. application on a mobile device, “WatchTV” activity is the most likely and therefore may be triggered, i.e.turn TV, STB (and AVR if needed) ON and set TV (and AVR if needed)inputs. A “Watch TV” activity may also be triggered when pressing abutton that is uniquely associated with the STB, PVR or DVR, e.g. GUIDE,LIST, DVR, THUMBS UP/DOWN, and the like. A “Play game” activity may betriggered when a user presses a button that is uniquely associated withthe game console, e.g. “GAME” (which is often a dedicated button toselect a specific TV input) on a TV. If the user has a STB and DVD, butno PVR (personal video recorder), pressing a transport key on the remotecontroller may trigger the “Watch Movie” activity. If the user has a PVRand DVD, pressing the a transport key on the remote controller maytrigger the “Watch TV” activity. Pressing the numeric keypad on a remotecontroller may trigger a “Watch TV” activity. Pressing the CHANNELUP/DOWN or PREV on a remote controller may trigger a “Watch TV”activity. Pressing the INPUT button on a remote controller may turn ONthe TV, invoke a TV UI application and open the TV Inputs menu in theUI. Pressing the FAVORITE button on the remote controller may trigger a“Watch TV” activity, invoke the TV UI and open the TV Favorites menu inthe TV UI. Pressing an OEM-specific button such as VIERCAST on a remotecontroller should turn ON the TV. Pressing a multiple system operation(MSO)-specific button such as VOD (video on demand) on the remotecontroller may trigger a “Watch TV” activity. Pressing the FAVORITEbutton on the remote controller may trigger a “Watch TV” activity.Pressing the FAVORITE button on the remote controller may trigger aspecial activity where only TV is turned ON and a UI combining shortcutsto various content items or sources may invoked.

As discussed above and elsewhere herein, the various UIs presented inaccordance with the present disclosure may take into account the contextof how a display device is being utilized. For example, a UI may beconfigured to only show DVD and AVR commands if currently in a Watch DVDactivity. As another example, UIs may be different, dependent on whethera user has upgraded to premium functionality. After the user upgrades topremium functionality, a remote control power button that typically onlyturns TV power on can trigger an application to turn the AVR and STB ON,and switch TV and AVR inputs. The same power button, in this example, isnow a “Watch TV” activity button. As yet another example, UI content canchange based on mode of operation. For instance UI may begin bypresenting user with an activities menu, such as an Activities UI, asabove. Once in an activity, a menu may show commands for devices in use.Alternatively, once in an activity, a UI may show access to sources ofinformation (from the an Internet-accessible information resource)related to the content currently consumed. For instance, when watching amovie, UI can provide quick access to various web sites dedicated tomovie content, reviews, and such. Access may be provided by providing anoverlay UI or a picture in picture showing director filmography, actors'bio, movie synopsis, user reviews, user ratings, and the like that havebeen accessed. The information source could also be an online servicethat has aggregated Internet searches that the community of users of theinvention has done over time around a given piece of content. Thisinformation may have been collected using IP controllers with browsingcapabilities, e.g. smartphone or tablet computing device, andtime-referenced with the show a corresponding user was watching. Otherexamples are considered as being within the scope of the presentdisclosure.

FIG. 7 shows an illustrative example of a process 700 for providing amenu, in accordance with an embodiment. The term menu may indicate a UIand/or collection of UIs, such as described above. Some or all of theprocess 700 (or any other processes described herein, or variationsand/or combinations thereof) may be performed under the control of oneor more computer systems configured with executable instructions and maybe implemented as code (e.g., executable instructions, one or morecomputer programs, or one or more applications) executing collectivelyon one or more processors, by hardware, or combinations thereof. One ormore of the actions depicted in FIG. 7 may be performed by a device,such as the display device 100, described above. Further, some or all ofthe process 700 may be performed collectively by multiple devices, suchas a bridge and a display device as described in application Ser. No.13/077,750, entitled “Apparatus and Method for Configuration andOperation of a Remote-Control System” filed on Mar. 31, 2011, which isincorporated herein by reference for all purposes. The code may bestored on a computer-readable storage medium, for example, in the formof a computer program comprising a plurality of instructions executableby one or more processors. The computer-readable storage medium may benon-transitory.

In various embodiments, the process 700 includes displaying 702 content.As noted, content may be displayed during performance of some or all ofthe process 700 and, therefore, displaying 702 content is shown as astep that encompasses other actions of the process 700. While otheractions may also be performed concurrently with yet other actions, forthe purpose of illustration, other actions are not illustrated as such.Displaying 702 the content may be performed in any suitable manner andthe content may be any content capable of being displayed including, butnot limited to, television content, content from a device such as a DVDor other media device, content streamed over the Internet or anothernetwork, electronic photographs, and other content.

In an embodiment, a signal indicating selection from a menu button on aremote control device, such as the remote control device 600 describedabove in connection with FIG. 6A, is received 704. The signal may bereceived directly by a display device or indirectly, such as by anotherdevice in communication with the display device. As one example, asignal may be received as a result of user interaction with a touchscreen interface. While the process 700 is described in connection witha menu button, other actions may be substituted in the process 700, orvariations thereof. For example, any suitable button may cause thesignal to be received. In an embodiment, many different buttons on aremote control device may cause a menu to appear. The menu that appearsmay depend on the button pressed. For example, if a user is utilizing anapplication of a content providing device and presses an EXIT button, anActivity UI may appear. Further, a conventional remote control deviceneed not be used in some embodiments. For instance, a camera may recordmotions of a user and, if a user makes a particular motion associatedwith the signal, the signal may be received. In such embodiments,navigation of menus may also proceed using sensed user motions, althoughsuch embodiments will not be described in detail. As another example, aremote control or other device may include one or more motion sensors,such as gyroscopic sensors and/or accelerometers. A signal may be sentfrom such a device responsive to movement of the device, perhaps after aperiod of not being moved or detecting a particular movement thatcorresponds to a specific gesture for causing the menu to appear. As anillustrative example, a remote control device may send a signal if oneor more sensors collectively indicate, for instance, that the remotecontrol device was picked up by a user.

When the signal is received 704, in an embodiment, a determination ofthe current mode of operation of the display device (and possibly otherdevices) is determined 706. The mode of operation may correspond to anactivity being performed, such as watching TV, watching a DVD, browsingthe Internet, and the like. The mode of operation may also include otherinformation, such as devices that are used to perform the activity. Forexample, one mode of operation may correspond to watching TV using aSTB, another mode of operation may correspond to watching TV streamedfrom an online source, and yet another mode of operation may correspondto watching TV from yet another source, such as an antenna that is usedto receive broadcast signals. The mode of operation may also includeinformation regarding any audio devices used to experience content.Generally, the mode of operation may correspond to any collection ofinformation regarding how a display device is being used to experiencecontent.

The process 700 may also include overlaying 708 a menu on the display,such as described above. The menu appearance may be based at least inpart on the determined mode of operation. For example, if it isdetermined that the current mode of operation corresponds to watching aDVD, a command UI, such as described above, may be overlaid on thedisplay. The command UI may be configured based on a current state ofthe activity being performed. For instance, if the DVD is paused, a playoption may be highlighted. In addition, the various options presentedmay depend on the state of the activity being performed. If the user isnavigating through interactive content of a DVD, the menu may includeoptions that are different from if the user is watching a movie encodedon the DVD. It should be noted that, in some embodiments, a menu may notbe overlaid on displayed content. For example, in other modes ofoperation (e.g., a “play music” activity), the menu may simply appear onthe display by itself, rather than as an overlay.

In an embodiment, a signal may be received 710. The signal may bereceived in any suitable manner, such as pursuant to selection of abutton on a remote control device or otherwise. It should be noted thatin some instances a signal may not be received at this point and theprocess 700 may deviate accordingly. For example, if the user doesnothing, a timeout may cause the overlaid menu to disappear. The menumay, for example, disappear after a period of time during which nosignals are received. Further, in some instances, a signal may not bereceived, such as if power to a device is lost, a device malfunctions,and the like. It should be noted that, as elsewhere herein, receipt of asignal may actually include receipt of multiple signals; however, forthe purpose of illustration, illustrative embodiments are described interms of a single signal.

When the signal is received, a determination is made 712 regarding aselection corresponding to the signal. For example, if it is determinedthat the received signal corresponds to selection of a menu button, theoverlaid menu may be removed 714 from the display. If, however, it isdetermined that the received signal corresponds to selection of anoption on a currently displayed menu, a corresponding function may beperformed. For example, if the option corresponds to an activity, if notalready in a suitable state, one or more devices may be put in asuitable state. If the activity is watching a DVD, for example, a DVDplayer and AVR may be put in a power-on state and configured asnecessary to allow the user to watch a DVD. Putting one or more devicesmay be performed by sending one or more appropriate signals to eachdevice, as necessary to put the devices in the correct state.Appropriate signals may be sent in any suitable manner. For instance,the display device may send signals to the other devices over an HDMIconnection, using infrared, or otherwise. As another example, a contentappliance such as the Logitech Revue (which may also act as a bridgedevice) may generate and send appropriate signals. A device generatingdisplay information (such as the display device itself or a deviceproviding the generated display information for the display device todisplay) may communicate with the content appliance or othersignal-generating device accordingly. As another example, if theselected option is navigational, the menu display may change accordingto how the user has navigated. A different option may be highlightedand/or a different menu may be overlaid on the display in addition orinstead of a currently-displayed menu. Generally, any suitable functionthat corresponds to the signal that was received may be performed.

FIG. 8 shows an illustrative example of a process 800 that may be usedto update a menu that has presented to a user on a display, such asdescribed above. For example, in an embodiment, the process 800 may becombined with the process 700 described above in connection with FIG. 7when a signal is received in connection with a displayed menu. In anembodiment, the process 800 includes receiving 802 a signal, such asdescribed above. A determination may be made 804 regarding an action towhich the signal corresponds. For example, in this illustrative example,a determination is made 804 regarding whether the signal corresponds tonavigation of the menu or selection of a function. If it is determinedthat the signal corresponds to navigation of the menu, anotherdetermination may be made 806 whether a new menu view should bepresented, where a new menu view may be, for example, a new UI, such asdescribed above. As just one example, referring to the illustrativeexample in FIG. 2, if it was determined that the user selected thenavigation bar 220, such as by selecting an up arrow on a remote controldevice when the navigation bar 220 was highlighted, it may be determinedthat an Activities UI should replace a currently displayed Device UI.If, however, it is determined, for example, that the signal correspondsto navigation from the DVD-1 icon to the AVR icon, or from the DVD-1icon to the navigation bar 230, it may be determined that a new menu isnot required.

Accordingly, in an embodiment, if it is determined 806 that a new menuview is required, a new menu may view may be overlaid 808 on thedisplay. Overlaying the new menu view may include replacing a currentlypresented menu view although overlaying the new menu can, in someembodiments, include adding a new menu view to a currently displayedmenu view. For example, referring to FIG. 2, in an embodiment, selectionof the menu bar 220 may cause the Activities UI to replace the Device UIthat is currently displayed. However, in some embodiments, theActivities UI may appear in addition to the Device UI, such as above theActivities UI. Returning to the process 800, if it is determined that anew menu view is not required, the currently displayed menu view may beupdated 810, such as by highlighting a new menu option. It should benoted that other actions are also possible, such as expanding expandablemenu options, displaying contents of a drop-down box, and, generally,updating a current menu view in any suitable manner that corresponds tothe signal that was received. Further, if the received signal did notcorrespond to anything that would require update of the menu view (or anew menu view), the current menu view may be left unchanged.

If it was determined 804 that the signal corresponds to a function to beperformed (such as if the user selected a menu option), then, in anembodiment, the function may be performed 812. If a device performingsome or all of the process 800 is to perform the function, the devicemay perform the function itself. If a different device is to perform thefunction, a signal may be sent to the device to instruct the device toperform the function. As an illustrative example, if a TV is performingthe process 800 and the signal corresponds to a function that the TVperforms (such as a volume level adjustment if the TV is providingaudio), the TV may perform the function. If, however, the TV isperforming the process 800 and the function corresponds to a DVD playerfunction (such as pause of DVD playback), a signal may be sent to theDVD player to cause the DVD player to perform the function. The signalmay be sent in any suitable manner, such as over an HDMI cable,wirelessly using IR or RF, or in another way.

In an embodiment, when the function is caused to be performed (or ifperformance of the function has at least been initiated), adetermination may be made 814 whether to remove the currently displayedmenu view. Some functions may be, for example, unlikely to require useof a menu once performance has been completed or at least initiatedwhile it may be more convenient for some functions to leave a menu viewdisplayed. For instance, subsequent to selection of a fast-forwardoption, it may be more convenient for the user to leave a menu viewdisplayed to allow the user to resume normal playback once content hasreached a desirable position without requiring the user to cause a menuview to appear again. However, subsequent to selection of an applicationoption, it may be more convenient to remove the current menu view toallow the user to utilize the interface of an application that haslaunched. Generally, each function may correspond to a determination 814whether to remove the current menu view. The correspondence may bepredetermined or may be learned over time, such as based on userbehavior or behavior of a collection of users.

Accordingly, if it is determined 814 to remove the menu view, the menuview may be removed 816 from a display, i.e., may be no longer overlaidon the display. As illustrated in FIG. 8, if it is not determined toremove the menu view, it may be determined 814 to update 818 the menuaccording to the function that as performed or to leave the menu view asit was. Determining whether to update the menu view or leave the menuview unchanged may be based at least in part on the function that wasperformed. For example, if the function corresponded to selection of apause option for a movie, the menu may be updated to display a playoption instead of the pause option that was selected. If the functioncorresponded to selection of a volume adjustment, as another example, itmay be determined to leave the menu view unchanged. As illustrated,whether or not the menu view is unchanged, the process 800 may repeat incertain circumstances. For example, if a signal is subsequentlyreceived, the process 800 may be performed again. Performance of theprocess 800 may be repeated, for example, as long as a user navigatesmenu views and/or until an event, such as a timeout, causes all menuviews to disappear from the display.

As noted, presentations of menus may vary based at least in part oncontext. FIG. 9, accordingly, shows an illustrative example of a process900 that may be used to provide a menu when a particular activity isbeing performed. In an embodiment, the process 900 includes receiving902 a signal, which may be a signal that indicates that a menu should beoverlaid on a display, such as described above. Upon receipt of thesignal, in an embodiment, the current activity is determined 904. Forexample, a determination may be made whether the activity is watching amovie, watching television, playing a game, interacting with anapplication, and the like. As above, while not illustrated as such, thedetermination may be more complex and may include determining whatdevices are being used and other relevant information.

The process 900, in an embodiment, also includes determining 906 acurrent state of the activity that is being performed. For example, ifthe activity is watching a movie, the state may be whether the movie isplaying or paused. If the activity is playing a game, the state may bewhether the game is being played or is paused, a level of the gamecurrently being played, and the like. Generally, the state may be anycollection of information about how an activity is currently beingperformed. In an embodiment, a menu is determined 908 based on thecurrent activity. A data store that associates activities with menus maybe accessed to make the determination. The menu may be determinedadditionally based on the state of the activity and/or otherinformation. For example, each activity/state pair of a set of possibleactivity/state pairs may correspond to a particular menu. (Somemenu/state pairs may correspond to the same menu.) Determining the menumay be performed in such an embodiment by identifying the menu from adata store that associates activities, states, and menus.

In an embodiment, the process 900 includes determining 910 an option ofthe determined menu to highlight. The determination may be made based atleast in part on the determined state. For example, if the activity iswatching television and the state is that content corresponds to acommercial break, a determination may be made to highlight an option forchanging a channel or, if a DVR is available for such use and thecurrent state is that the current content is pre-recorded on the DVR, itmay be determined to highlight a fast-forward button to allow the userto fast-forward through the commercial break. Generally, any suitablemanner of determining the option to highlight may be used. When theoption to be highlighted is determined, in an embodiment, the determinedmenu with the highlighted option is provided 912. Providing the menu maybe performed in any suitable manner, such as by overlaying the menu on adisplay, sending information to a display device to enable the displaydevice to overlay the menu, and the like.

FIG. 10 shows an illustrative example of a process 1000 for determiningan option to highlight, which may be part of the process 900 describedabove in connection with FIG. 9 in a specific circumstance, i.e. when amovie is being watched. In an embodiment, the process 1000 includesdetermining 1002 a current activity such as watching a movie. Adetermination is then made 1004 whether the state of watching the movieis that the movie is playing or the movie is paused. It should be notedthat the process 1000 is simplified for the purpose of illustration andthat other states may be possible. If it is determined 1004 that thestate is that the movie is playing, a highlighted option is set 1006 toa pause option. If, however, it is determined 1004 that the state isthat the movie is paused, a highlighted option may be set to a playoption. As noted above, a menu with the determined option highlightedmay be presented in manner that results in a menu with the highlightedoption being overlaid on a display. In this manner, a user is able toimmediately select “play” when a movie is paused and immediately select“pause” when the movie is playing. Therefore, in many instances, theuser is allowed to select an option without unnecessary navigationalsteps.

FIG. 11 shows an illustrative example of a process 1100 for determiningan option to highlight, which may be part of the process 900 describedabove in connection with FIG. 9 in a specific circumstance, i.e. when amovie is being watched. In this example, the process 1100 includesdetermining 1102 the current activity as watching television. Adetermination may be made 1104 whether a DVR is being used and/or isavailable for use. If it is determined 1104 that a DVR is being usedand/or available for use, a determination may be made 1106 whether thecurrent content being displayed is pre-recorded or live, wherepre-recorded indicates that the content is being played from memory ofthe DVR rather than, for example, a STB or an antenna. If it isdetermined 1106 that the content being watched is pre-recorded, thehighlighted option of a DVR menu may be set 1108 to fast-forward. Itshould be noted that, while not illustrated as such, the process 1100may include determining which menu to display, such as described above.If, however, it is determined that the content being watched is notpre-recorded, the highlighted option may of a DVR menu be set 1110 to apause option.

In an embodiment, if it is determined 1104 that a DVR is not used oravailable for use, a determination may be made whether the currentcontent is an advertisement 1112. The determination may be made in anysuitable manner. For example, a content provider, such as a satellitetelevision provider, a broadcast television provider, and the like, mayincorporate signals into their broadcasts that identify whether currentcontent is an advertisement. Such signals may be used to make adetermination. Other techniques may also be used to determine whethercurrent content is an advertisement. For instance, image recognitiontechniques may be used to compare one or more screenshots with one ormore screenshots of known advertisements in a remote or other database.If a screenshot matches an image in such a database, a determination maybe made that the current content is an advertisement. The Internet orother communications network may be used to access such a database andscreenshots may be downsampled to decrease latency. Further, screenshotsmay be periodically checked against a remote database so that, at thepoint a determination needs to be made, a recent screenshot has alreadybeen compared. As with other actions discussed herein, a determinationwhether current content is an advertisement may be made at a timedifferent from that which is implied by the illustrative example of thefigure. Of course, any suitable technique to make the determination maybe used.

If it is determined that the current content is an advertisement, in anembodiment, a determination is made 1114 whether to highlight a “channelup” option or a “channel down” option (or another option, such as a“last channel watched” option). The determination may be based at leastin part on the current state of the activity. For example, if thecurrently watched channel corresponds to a high number, thedetermination may be to highlight the channel down option. Similarly, ifthe currently watched channel corresponds to a low number (such as thefirst channel in a sequence of channels), the determination may be tohighlight the channel up option. The determination may also be based atleast in part on recorded user behavior. For example, channels watchedby a user (or group of users, such as a household) may be recorded. Ifmore channels that the user typically watches are below the currentchannel, the determination may be made to highlight the channel downoption. Factors such as the time spent on each channel, the time of daythe channel is currently watched, and the current time may be factoredinto a determination.

In the illustrative example of FIG. 11, once it has been determinedwhether to highlight the channel up option or the channel down option,the channel down option is set 1116 to be highlighted 1116 or thechannel up option is set 1118 to be highlighted according to thedetermination. If however, it has been determined that the currentcontent is not an advertisement, in an embodiment, a different option isset to be highlighted. For example, as illustrated in FIG. 11, in anembodiment, a mute option is set 1120 to be highlighted. Of course, theparticular determinations and options set to be highlighted are providedfor the purpose of illustration and different embodiments may performthe process 1100 in a different manner, with different options.

FIG. 12 shows an illustrative example of a process 1200 for providing anactivity menu, such as the Activity UI described above. In anembodiment, the process 1200 includes receiving an indication of anactivity menu selection by the user. For example, the user may havenavigated to an Activity UI, as described above. As another example, auser may have selected the menu button on a remote control device, suchas described above, and a corresponding signal may have been received.As a result of the signal being received, context may have been used todetermine that an activity menu should be displayed in response to theuser having selected the menu button.

In an embodiment, a set of possible activities is determined 1204. Theset of possible activities may be determined in any suitable manner. Forexample, a set of activities may be a set of activities that a user hasconfigured through a configuration process. A set of activities may alsobe determined based at least in part on what activities are possiblebased on a set of devices that are available for use with activities.For example, if a DVD player is not available to play a DVD, thedetermined set of activities may exclude watching a DVD. Determining aset of activities may also include selecting a subset of a larger set ofactivities. For example, if a menu can only display some number of alarger number of possible activities, the set may be determined byselecting from the possible activities.

As illustrated in FIG. 12, an ordering for the determined set ofactivities may be determined 1206. The ordering may be based at least inpart on various information available, such as recorded user behavior,time of day, and the like. For example, the activities may be orderedbased at least in part on records regarding activities that have beenselected in the past. For example, the activities may be ordered basedat least in part on a recorded frequency of selection over a timeperiod. The frequency may be measured for a particular part of the dayand the ordering may be based at least in part on a frequency for acurrent time of day. In addition, past user behavior may be used toorder the activities. As one example, if a user frequently watches aparticular television program on Thursdays at 9:00 PM, a WatchTelevision Activity may occur first (or near first) in a determinedordering. Of course, other methods for ordering the activities may beused, including alphabetical ordering, random ordering, and/or othermethods.

Also illustrated in FIG. 12, the process 1200 may include determining anactivity to highlight. For example, it may be determined to highlightthe first of an ordered set of activities. It should be noted that thefirst of an ordered set of activities may not necessarily correspond tothe first in a list of activities as the list appears on a menu. Forexample, if a set of activities is to be displayed as sequence of iconson a menu, the first in the ordering may correspond to a prominentposition in the sequence (such as the middle) and not necessarily theleftmost activity. The determination may also be made in other ways,such as described above. For example, a determination may be made as tothe activity that the user is most likely to select, which may be madebased at least in part on past recorded behavior of the user and/or of agroup of users. Once all applicable determinations have been made, in anembodiment, the activity menu may be overlaid 1210 on a displayaccording to the ordering with the highlighted option, such as describedabove.

In some instances, when a user selects an activity option of a menu,there may be several options for performing that activity. Televisioncontent, for example, can be obtained from various devices, such assatellite STBs, cable STBs, fiber optic STBs, antennas, remote servers,etc. FIG. 13, accordingly, shows an illustrative example of a process1300 that may be used when a user selects an activity of a menu, such asan Activity UI, as described above. In an embodiment, the process 1300includes receiving 1302 an indication of selection of an activity by auser, such as described above. A determination may be made 1304 of thepossible modes of the activity that were selected. For example, if theuser selects a Watch TV option, a determination may be made as to thepossible content sources for television content. Similarly, if the userselects a Watch a Movie option, a determination may be made regardinghow the user can watch a movie (e.g., using a DVD player, a Blu-Rayplayer, a video streaming service, an online subscription service suchas Netflix®, movies currently playing in a television broadcast, moviespre-recorded on a DVR, and the like. Sources of movie content mayinclude accessible metadata, such as tags, that may be used to identifythe content as a movie). The determination may be made based at least inpart on one or more devices in communication with a device being used inconnection with performance of the process 1300. For example, if anInternet connection is not currently available, a video streamingservice mode may be excluded from a set of possible activity modes. Aswith other determinations, the determination, in some embodiments, doesnot need to be made responsive to selection of an activity, but may havebeen made in a background process prior to user selection of theactivity in order to improve UI responsiveness.

In an embodiment, the determined modes for the selected activities areordered 1306, such as in a manner described above and a determination ismade 1308 as to which option to highlight. As with other processesdescribed above, once all appropriate determinations have been made, amenu of activity modes may be overlaid 1310 on a display with theappropriate highlighted option. The menu may replace a menu ofactivities, such as the Activity UI above, or may be presented in otherways.

As with any menu described herein, a menu option may be selected by auser. FIG. 14 accordingly shows an illustrative example of a process1400 that may be performed when a user selects a mode of an activity,such as in connection with the process 1300 of FIG. 13 described above.It should be noted that, in some embodiments, an activity mode may beselected in other ways. For example, some activities may have only onemode for a given configuration and, selecting the activity mode may beperformed by virtue of selecting the corresponding activity.

As illustrated in FIG. 14, in an embodiment, an indication of selectionof an activity mode is received 1402, such as in a manner describedabove. As discussed, a mode of an activity may correspond to use of aplurality of devices, such as an AVR, a DVD player, a STB, a TV, and/orother devices. Accordingly, in an embodiment, the process 1400 includesdetermining 1404 a set of devices associated with the selected activitymode. For instance, if the selected activity mode corresponds towatching a DVD, the set of devices may include a TV, a DVD player, andpossibly an AVR or other system for providing audio.

In an embodiment, the selected activity mode may require each of thedetermined devices to be in a particular state. For instance, watching aDVD may require a TV to be set to an input corresponding to a DVDplayer. The DVD player may need to be put into a state where the DVDplayer reads content from a DVD. An AVR, if used, may need to be put ina state where audio signals are received form the DVD player.Accordingly, in an embodiment, the process 1400 includes determining arequired state for a first device for the selected activity mode. Adetermination may then be mode 1408 whether to update the state of thedevice. The determination may be made in any suitable manner. Forexample, the state of the device can be tracked as a user utilizesvarious devices and determined based on a tracked state of the device.Whether to change the state of the device can be determined bycommunicating with the device, such as through an HDMI cable connectedto the device. In addition, determining whether to change the state ofthe device can be made by, for example, assuming that the device is in aparticular state and determining whether the particular state differsfrom the state the device needs to be in. For example, the determinationmay be made for a DVD player by assuming that the DVD player is in apower-off state.

When it is determined whether to update the state of the first device,in an embodiment, one or more signals are determined 1410 for puttingthe device in the proper state. For example, a signal corresponding to apower-on state may be determined. If more than one signal is needed toput the device in the correct state, multiple signals may be determined.In any event, one or more signals may then be transmitted 1412 to thedevice for which the signals were intended. The device may receive thesignal and change state accordingly, but not necessarily so. Forexample, a DVD player already in a power-on state may do nothing uponreceipt of a power-on signal.

A determination may then be made whether there are additional devicesand portions of the process may be performed again until all devices areput into the correct state, as illustrated in the figure. It should benoted that, as with all processes described herein, variations areconsidered as being within the scope of the present disclosure. Forexample, the process 1400 may include determining all signals requiredto put devices in the proper state before sending all of the determinedsignals to the respective devices.

Generally, other variations are also considered as being within thescope of the present disclosure. For example, FIGS. 2-5 provideillustrative examples of menu layouts for the purpose of illustration.Generally, layouts may differ substantially from that which isillustrated. In an embodiment, a UI is designed such that everysupported device or activity is visible as a menu option, and such thatnavigation steps to reach an option are, at least in some instances,minimized. For instance, in an embodiment, menu options can be scrolledthrough continuously in either direction (i.e. as a “carousel”). Thesequence of device menus (TV—STB—AVR—DVD) may be arranged as a carouselthat is defined such that the minimum of steps are required to manualengage into an activity such as watching live TV or watching a movie.The number of menu options may be minimized. For instance, in anembodiment, the UI only presents only three items (devices commands) atmost when a group of commands is selected in order to minimize the TVreal estate occupied by the UI, although different numbers may be usedin different embodiments. Cues that more items are available in the listmay be provided, such as icons, labels, color gradients, font colors,and the like. When a user navigates to a group of device commands, themenu of commands may, in some instances, automatically appear withoutthe user pressing the “OK” button or otherwise actively making aselection. When a cursor or other indication of immediately selectableoption is in a menu of commands, in an embodiment, the user can proceedto the next group of commands by simply moving left or right (no need toscroll down and back to the main menu line). When navigating to a groupof commands, the UI element representing the group may be replaced withthe most frequently used command in order to minimize the number ofsteps. When opening up a group of device commands, the cursor or otherindicator of an immediately selectable option may automatically moves tothe most frequently used command, e.g. “Guide” for a “Live TV” group ina STB menu.

Further, embodiments of the present disclosure also allow for menucustomization by a user. For instance, a setup process may be invoked(either initially or at some later time) to allow a user to customize UIpresentations. During such a setup process, the user may be able tochange the order of groups and commands in a device menu or the order ofcommands within a menu. The same could be done from a settings menu. Theorder of command within a group may be ordered alphabetically bydefault, or based on the most common order in the population of users(crowd sourcing), and the like. The order of commands within a groupand/or he order of groups & commands within a menu may be optimizedautomatically based on the usage tracking of a specific user, in orderto minimize travel when navigating the UI. At various time, the user maybe able add a currently watched television channel or other contentsource indicators to a maintained list of favorites by pressing andholding a dedicated menu button on his/her remote control device, or bytaking some other action. At various times, the user may be able to adda URL of a currently viewed web page to a list of shortcuts by pressingand holding the dedicated menu button on his/her remote. At varioustimes the user may be able to add the application (currently in focus orrunning) to the list of shortcuts by pressing and holding the dedicatedmenu button on his/her remote. Other variations in addition to thoseexplicitly described herein are also considered as being within thescope of the present disclosure.

In addition, embodiments of the present invention further includecomputer-readable storage media that include program instructions forperforming various computer-implemented operations as described herein.The computer readable medium is any data storage device that can storedata which can thereafter be read by a electronic system. The media mayalso include, alone or in combination with the program instructions,data files, data structures, tables, and the like. The media and programinstructions may be those specially designed and constructed for thepurposes of the present subject matter, or they may be of the kindavailable to those having skill in the computer software arts. Examplesof computer-readable storage media include magnetic media such as flashdrives, hard disks, floppy disks, and magnetic tape; optical media suchas CD-ROM disks; magneto-optical media such as floptical disks; andhardware devices that are specially configured to store and performprogram instructions, such as read-only memory devices (ROM) and randomaccess memory (RAM). Examples of program instructions include bothmachine code, such as produced by a compiler, and files containinghigher level code that may be executed by the computer using aninterpreter.

The computer readable medium can also be distributed over a networkcoupled to electronic systems so that the computer readable code isstored and executed in a distributed fashion.

Other variations include, but are not limited to, variations on thehardware used to implement various embodiments of the presentdisclosure. For example, as discussed above, a bridge device may be usedto receive signals form a remote control device based on userinteraction with the remote control device and transmit signals to oneor more other consumer electronic devices accordingly. Bridge devicesmay be configured in numerous ways. For example, in various embodiments,a bridge device has a single video input and a single video output to adisplay device. In this embodiment, the bridge device may receive avideo signal through the video input and output a video signalaccordingly. Instead of the display device overlaying menus on thedisplay, the bridge device may modify the video signal to cause thedisplay device to display menus generated in accordance with variousembodiments. In this manner, the display device does not requirecomputing logic for generating and displaying menus other than to decodethe incoming video signal. As another example, the bridge device mayinclude multiple video inputs and one or more video outputs. The bridgedevice may select from the inputs for receipt of a correspondingincoming video signal. Selection of the input may be made according touser input with a remote control device. For example, if a user selectsa “Watch DVD” activity using a remote control device, the bridge devicemay select an input that receives a signal from a DVD player. A outgoingvideo signal may be sent to a display device accordingly and theoutgoing video signal may be modified to overlay various menus on thedisplay.

The description given above is merely illustrative and is not meant tobe an exhaustive list of all possible embodiments, applications ormodifications of the invention. Thus, various modifications andvariations of the described methods and systems of the invention will beapparent to those skilled in the art without departing from the scopeand spirit of the invention. Although the invention has been describedin connection with specific embodiments, it should be understood thatthe invention as claimed should not be unduly limited to such specificembodiments.

Other variations are within the spirit of the present invention. Thus,while the invention is susceptible to various modifications andalternative constructions, certain illustrated embodiments thereof areshown in the drawings and have been described above in detail. It shouldbe understood, however, that there is no intention to limit theinvention to the specific form or forms disclosed, but on the contrary,the intention is to cover all modifications, alternative constructions,and equivalents falling within the spirit and scope of the invention, asdefined in the appended claims.

The use of the terms “a” and “an” and “the” and similar referents in thecontext of describing the invention (especially in the context of thefollowing claims) are to be construed to cover both the singular and theplural, unless otherwise indicated herein or clearly contradicted bycontext. The terms “comprising,” “having,” “including,” and “containing”are to be construed as open-ended terms (i.e., meaning “including, butnot limited to,”) unless otherwise noted. The term “connected” is to beconstrued as partly or wholly contained within, attached to, or joinedtogether, even if there is something intervening. Recitation of rangesof values herein are merely intended to serve as a shorthand method ofreferring individually to each separate value falling within the range,unless otherwise indicated herein, and each separate value isincorporated into the specification as if it were individually recitedherein. All methods described herein can be performed in any suitableorder unless otherwise indicated herein or otherwise clearlycontradicted by context. The use of any and all examples, or exemplarylanguage (e.g., “such as”) provided herein, is intended merely to betterilluminate embodiments of the invention and does not pose a limitationon the scope of the invention unless otherwise claimed. No language inthe specification should be construed as indicating any non-claimedelement as essential to the practice of the invention.

Preferred embodiments of this invention are described herein, includingthe best mode known to the inventors for carrying out the invention.Variations of those preferred embodiments may become apparent to thoseof ordinary skill in the art upon reading the foregoing description. Theinventors expect skilled artisans to employ such variations asappropriate, and the inventors intend for the invention to be practicedotherwise than as specifically described herein. Accordingly, thisinvention includes all modifications and equivalents of the subjectmatter recited in the claims appended hereto as permitted by applicablelaw. Moreover, any combination of the above-described elements in allpossible variations thereof is encompassed by the invention unlessotherwise indicated herein or otherwise clearly contradicted by context.

All references, including publications, patent applications, andpatents, cited herein are hereby incorporated by reference to the sameextent as if each reference were individually and specifically indicatedto be incorporated by reference and were set forth in its entiretyherein.

What is claimed is:
 1. A remote control system, comprising: a controllerincluding a plurality of fixed-function buttons and permanentlyconfigured to output a plurality of command signals corresponding to theplurality of buttons; and a hub, said hub including one or moreprocessors; and memory including instructions that, when executed by theone or more processors, cause the remote control system to at least:determine a menu that includes one or more options for controlling atleast one consumer electronic device from a set of consumer electronicdevices; in response to receipt of an indication of user intent to viewthe menu, including one or more of the command signals from thecontroller, cause the determined menu to appear on a display of a devicethat is different from the consumer electronic device; receive one ormore of the command signals corresponding to user interaction with thedisplayed menu; and when the user selects an option of the menu forcontrolling the consumer electronic device, causing a signal to betransmitted to the consumer electronic device, the signal beingconfigured to cause the consumer electronic device to change stateaccording to the selected option, wherein, each of the plurality ofbuttons is permanently configured to activate its corresponding commandsignal regardless of the state of the consumer electronic device.
 2. Theremote control system of claim 1, wherein the device that is differentfrom the consumer electronic device is a television.
 3. The remotecontrol system of claim 1, wherein causing the determined menu to appearon the display includes modifying the display such that the menu isoverlaid atop content of the display.
 4. The remote control system ofclaim 1, wherein the menu includes at least one selectable option for afunction that is incapable of being selected using a dedicated input ofthe remote control device.
 5. The remote control system of claim 1,wherein determining the menu is based at least in part on a currentstate of the consumer electronic device.
 6. The remote control system ofclaim 1, wherein the menu further includes one or more options forcontrolling at least one other device.
 7. The remote control system ofclaim 6, wherein the other device is a television.
 8. A remote controlconfiguration system, comprising: a display device including: a videodisplay; a processor configured to interact with a remote control; acommunication apparatus configured for communicating with at least oneother device; wherein, the processor is further configured to: present auser interface including selectable commands on the display; receive aset of predetermined commands from the remote control; interpret atleast one of the predetermined commands as a navigation command in onecontext and as a received command in another context; communicate thereceived command to the at least one other device; and modify the userinterface based on the navigation command.
 9. The system of claim 8,wherein the user interface is determined based at least in part on thestate of a set of devices that includes at least one other devicedifferent from the display device.
 10. The system of claim 8, whereinthe display device is a TV.
 11. The system of claim 8, wherein theprocessor is further configured to make an inference of relevantactivities for the other device based on a current mode of operation ofthe display device.
 12. The system of claim 11, wherein the selectablecommands presented by the user interface depend at least on adetermination of whether the other device is currently in use.
 13. Thesystem of claim 8, wherein the processor is further configured to atleast one of activate and update the user interface for the other devicebased on automated network communication with a remote service provider.14. The system of claim 8, wherein the user interface is configured toallow a user, while watching the display device operating in a firstmode of operation, to launch an application operable in a second mode ofoperation.
 15. The system of claim 8, wherein the user interface isconfigured to allow a user, while viewing content on the display device,to accept content-related commands for the other device, and tocommunicate the content-related commands to the other device, thecontent-related commands communicated to the other device including atleast one data element with information based on the content.
 16. Thesystem of claim 8, wherein the user interface is configured to detect acurrent operational state of the other device and to change the userinterface for the other device based on the operational state of theother device.
 17. The system of claim 8, wherein the user interfaceincludes a user-selectable and user-definable search feature that isoperable to search at least one of content stored on the other device,content available from a remote on-demand service, and a scheduledprogramming service.
 18. A method of controlling a peripheral device viaa display device comprising: controlling the display device via a hub topresent a first user interface including a plurality of selectableoptions; while presenting the first user interface on the displaydevice, receiving a control signal via the hub and interpreting thecontrol signal as a selected option of one of the plurality ofselectable options; communicating, based at least in part on theselected option, commands from the hub to the peripheral device; whilenot presenting the first user interface on the display device, receivingthe control signal via the hub and interpreting the control signal as aperipheral device command; and communicating the peripheral devicecommand from the hub to the peripheral device.
 19. The method of claim18, wherein presenting the user interface on the display device is basedat least in part on the state of a set of devices that includes theperipheral device, the peripheral device being different from thedisplay device.
 20. The method of claim 18, further comprising changingthe selectable options based at least in part on a current mode ofoperation of the display device.
 21. The method of claim 20, wherein:the display device includes selectable modes of operation including atleast three of tuner, AV, HDMI, S-Video, web-streaming, and web browsingmodes; the current mode of operations is selected from the modes ofoperation; and different selectable commands are displayed in the userinterface based on the at least three modes of operation.
 22. The methodof claim 18, further comprising automatically determining whether theperipheral device is ON or OFF, and only presenting the first userinterface for the peripheral device if the peripheral device is ON. 23.The method of claim 18, further comprising automatically determiningwhether the peripheral device is in use and making an automatedinference of relevant activities for the peripheral device based on acurrent mode of operation of the display device.
 24. The method of claim18, wherein the selectable options presented by the first user interfacedepend on the determination of whether the peripheral device iscurrently in use.
 25. The method of claim 18, further comprising atleast one of activating and updating the first user interface for theperipheral device based on an automated network communication with aremote service provider.
 26. The method of claim 18, further comprisingallowing a user, while watching the display device operating in a firstmode of operation, to launch an application operable in a second mode ofoperation.
 27. The method of claim 18, further comprising: receivingfrom a user, while the display device is displaying content,content-related commands for the peripheral device; and communicatingthe content-related commands to the peripheral device, thecontent-related commands communicated to the peripheral device includingat least one data element with information based on the content.
 28. Themethod of claim 18, further comprising detecting a current operationalstate of the peripheral device and changing the first user interface forthe peripheral device to a second user interface based on theoperational state of the peripheral device.
 29. The method of claim 18,further comprising presenting, as part of the first user interface, auser-selectable and user-definable search feature that is operable tosearch at least content stored on the peripheral device, contentavailable from a remote on-demand service, and a scheduled programmingservice.
 30. A method of controlling a set of one or more consumerelectronic devices comprising: under the control of a set of one or morecontrolling components of a remote control system, determining a menuthat includes one or more options for controlling at least one consumerelectronic device from the set of consumer electronic devices; causingthe determined menu to appear on a display of a device that is differentfrom the consumer electronic device; receiving one or more signalscorresponding to user interaction with the displayed menu, the signalshaving been generated remotely from the device having the display; andwhen the user selects an option of the menu for controlling the consumerelectronic device, causing a signal to be transmitted to the consumerelectronic device, the signal being configured to cause the consumerelectronic device to change state according to the selected option;wherein the set of one or more controlling components include a remotecontrol device and a bridge device, the remote control device includinga plurality of fixed-function buttons and permanently configured tooutput a plurality of command signals corresponding to the plurality ofbuttons regardless of the state of the consumer electronic device; thereceived one or more signals are generated by the remote control deviceaccording to user interaction with the remote control device; and thebridge device generates and transmits the signal to the consumerelectronic device.
 31. The method of claim 30, wherein the displaydisplays content according to a video signal from the bridge device andwherein causing the determined menu to appear on the display includesmodifying the video signal.
 32. The method of claim 31, wherein thebridge device includes a plurality of video inputs, wherein the videosignal is based at least in part on a signal received through aparticular video input of the plurality of video inputs, and wherein thebridge device is configured to select from the plurality of video inputsbased at least in part on user input to the remote control device. 33.The remote control system of claim 1, wherein the hub is configured tointerpret at least one of the command signals as a navigation command,which causes navigation of the displayed menu, in one context, and as areceived command, that instructs an operation of the at least oneconsumer electronic device, in another context.